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DETAILED ACTION 
Continued Examination Under 37 CFR LI 14 
1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1 .17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1 .17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.114. Applicant's submission filed on 29 March 2006 has been entered (Applicant's 
remarks were filed on 27 June 2005). 



Response to Amendments and Arguments 
2. Applicant has amended independent claims 24 and 33 to denote that "at least three 
image-acquiring sensor areas are located on one or more imaging sensors," rather than two , as 
previously claimed. 

This amendment is sufficient to overcome the previous § 102(b) rejections in view of 
Waxman (5,555,324). However, the claims are considered an obvious variant of Waxman and 
Watkins (6,597,807), as explained below. [Watkins was previously cited on a PTO-892 by the 
Examiner.] Whereas Waxman utilizes two "image- acquiring sensor areas" corresponding to 
different spectral bands, Watkins utilizes three such sensor areas. As shown in figure 1 of 
Watkins, each of a pair of stereo cameras contains three different "sensor areas" that are sensitive 
to different spectral bands. The images obtained by the three sensors on each camera are 
assigned RGB color codes and fused together into a single composite image, thereby producing 
stereo RGB composite images to be displayed e.g. to the left and right eyes of a viewer. 
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It would have been obvious to utilize three different spectral sensors in a multi-spectral 
imagaing system, instead of two as taught by Waxman, for the reasons provided below. 

Claim Objections 

3. Claim 30 is objected to under 37 CFR 1 .75(c), as being of improper dependent form for 
failing to further limit the subject matter of a previous claim. Applicant is required to cancel the 
claim(s), or amend the claim(s) to place the claim(s) in proper dependent form, or rewrite the 
claim(s) in independent form. 

Claim 30 provides for "one or more additional sensors." This does not further limit parent 
claim 24, which calls for the boundless, "one or more imaging sensors." Only if claim 24 were 
changed to recite a discreet or maximal number of sensors would claim 30 be further limiting. 

4. Claim 3 1 is objected to under 37 CFR 1 .75(c) for improperly referring to "said sensors" 
in claim 24, which provides that there may only be one sensor. Suggestion correction is to 
change "said sensors comprise" to — said one or more imaging sensors each comprise --. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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6. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

7. Claims 24-28 and 30-40 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 5,555,324 by Waxman et al. ("Waxman") in view of U.S. Patent 6,597,807 by 
Watkins et al. ("Watkins") and U.S. Patent 5,581,638 by Givens et al. ("Givens"). 

Regarding claims 24 and 33, Waxman discloses an image processing apparatus/method 
(figure 4) for processing imaging data in a plurality of spectral bands and fusing the data into a 
color image, comprising: 

two imaging sensors (cameras 310 and 312); 

at least two image-acquiring sensor areas located on said two imaging sensors (i.e. a first 
sensor area on camera 310 and a second sensor area on camera 312), wherein each said sensor is 
sensitive to a different spectral band than at least one other of said sensor areas (sensor area for 
camera 310 is sensitive to visible-near IR spectral band, and sensor area for camera 312 is 
sensitive to long-wave IR spectral band) and generates an image output representative of an 
acquired image in the spectral band to which the sensor area is sensitive (sensors for cameras 310 
and 312 produce visible/near IR and long-wave IR images, respectively); 
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a frame grabber connected to said imaging sensors (A/D converters 328 digitize and 
transmit frames of image data to the processors); 

a general purpose computer (processor 330 is a general-purpose computer that performs 
various functions) connected to said imaging sensors for executing in real time: 

a registration algorithm registering said image outputs (vision processor 392 contains 
processors 360 and 362 (see fig. 6), which execute algorithms registering and generally 
preprocessing the image outputs); and 

a color fusion algorithm for combining said image outputs into a single image (fig. 6 
shows the execution of the algorithm for combining the VIS-NIR and LWIR image signals into a 
single RGB image). 

Waxman does not expressly teach that there are three image-acquiring sensor areas, as 
claimed. As shown in figure 4, Waxman utilizes a pair of image sensors, wherein each sensor is 
sensitive to a different spectral band. [The cameras share the same field of view and are focused 
along a common axis - col. 10/42-45.] 

Therefore, in Waxman's system, there are only two "image-acquiring sensor areas * * * 
sensitive to a different spectral band," as claimed. 

Watkins discloses an image fusion system (figure 1) that is similar to Waxman's system 
Like Waxman, Watkins utilizes cameras that are sensitive to more than one spectral band. The 
different spectral images captured from each sensor are fused into a single image according to 
RGB color codes. Then, the fused images of each stereo camera are displayed to e.g. the left and 
right eyes of a user - see col. 4/35-48. 
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But whereas Waxman's cameras are sensitive in only two different spectral bands (VIS- 
NIR & LWIR), Watkins's cameras are each sensitive to three spectral bands (Near-IR, Mid-IR, 
and Far-IR). Watkins's sensors thus contain "three image-acquiring sensor areas * * * sensitive to 
a different spectral band." 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Waxman by Watkins to achieve the claimed invention by employing three image- 
acquiring sensor areas sensitive to different spectral bands, as taught by Watkins. Watkins 
teaches that acquiring three different spectral images and then fusing those images together 
according to color for display to a user provides a representation of a viewed scene that 
capitalizes on the properties of the human visual system to offer enhanced target recognition 
capabilities. Specifically, capturing and displaying three different spectral images in the manner 
taught by Watkins takes advantage of a human's preattentive vision characteristics to allow 
objects in a scene to be identified more easily. See column 3/25-53 and column 4/48-67. 

While Waxman teaches that the registration algorithm performed by preprocessors 360 
and 362 compensates for alignment, distortion errors, and the like (see column 13/37-40), 
Waxman does not expressly teach that the algorithm involves "scaling the image outputs," as 
claimed. 

Givens discloses a method for autonomous image registration for aligning two images 
having a common coverage area, such as those images obtained by Waxman in figure 6. In 
particular, Givens teaches that the process of registering two images having a common coverage 
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area conventionally involves scaling down the images (see column 5, lines 36-45: the images are 
resampled at 1/8X magnification and then minified at a magnification of 1/32X magnification). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Waxman and Watkins by Givens to achieve the claimed invention by including a 
scaling operation of the images in the registration algorithm, as claimed, since Givens teaches 
that scaling common-coverage area images to be registered was a conventional image processing 
technique employed during an image registration process in order to reduce the images to a very 
low resolution (see column 5/36-45). The registration operations subsequently performed on the 
images reduced by 32X would then be much less computationally intensive than if those 
operations were performed on the full resolution images. 

Regarding claim 25 and 34, Waxman discloses a screen display to display the outputs 
(332, figure 4). 

Regarding claims 26 and 35, Waxman discloses an operator interface for allowing 
operator input in processing of said image outputs (column 15, lines 33-37: user input is allowed 
for selecting a color mapping). 

Regarding claims 27 and 36, the combination of Waxman and Watkins teaches the color 
fusion algorithm is simple color fusion whereby each sensor area image output is separately 
assigned to a different color based on wavelength (Waxman, figure 6: colors are assigned on the 
basis of the wavelegnth being VIS-NIR or LWIR; and Watkins teaches that for each of the three 
different spectral images, RGB color codes are assigned based on the wavelengths thereof to 
create a single fused color image). 
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Regarding claims 28 and 37, the combination of Waxman and Watkins teaches the color 
fusion algorithm is based on principle component color fusion whereby said sensor area outputs 
are fused into one image (Waxman, column 3, lines 28-43: the assigned RGB color codes are 
converted to the HVS color space to form a single, color- fused image). 

Regarding claim 30, the combination of Waxman and Watkins teaches using an 
additional sensor having some of the sensors areas (Watkins's camera 310, in addition to camera 
312; and Watkins's stereo cameras). 

Regarding claims 31 and 38, Watkins teaches that each sensor comprises three sensors, 
each configured to map its image to an associated color channel (see figure 1), and a fusion 
algorithm is configured to combine the color channels into a color image (see figure 1 : the color 
channels for each stereo camera are combined into right and left composite video signals). 

Regarding claims 32 and 39, Watkins teaches the three sensors are sensitive to NIR, 
MWIR, and LWIR spectral bands (see figure 1). 

Regarding claim 40, the combination of Waxman and Watkins teaches the processing and 
fusing of said image occurs in real time (Waxman, column 1 1, lines 1-4). 



Claim Rejections - 35 USC §103 
8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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9. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1 .56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

10. Claim 29 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. Patent 
5,555,324 by Waxman et al. ("Waxman") in view of U.S. Patent 6,597,807 by Watkins et al 
("Watkins") and U.S. Patent 5,581,638 by Givens et al. ("Givens"), as applied to claim 28, and 
further in view of U.S. Patent 4,533,938 by Hurst. 

Regarding claim 29, Waxman discloses performing a PCCF algorithm (i.e. converting 
RGB to HVS values) in order to perform "desirable color manipulations" within the HVS space 
(column 15, lines 5-21). Processor 520, figure 10, is capable of performing color transformations 
to produce desirable effects on the image to be displayed. 

Waxman does not disclose desaturating the HVS image, since Waxman' s shunting 
algorithms produce images that are typically unsaturated (column 8, lines 48-51). However, 
should the final color image be saturated, it would have obvious to utilize processor 520 to 
desaturate the HVS image using any of the well-known desaturation techniques in order to 
enhance the image for display. 

For example, Hurst discloses an image processing system wherein a desaturation signal is 
added to the image signal when a hue value is modified (see Abstract). This desaturation signal 
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is added in order to minimize the changes in saturation when the hue is modified. It would have 
been obvious to one of ordinary skill in the art at the time of the invention to modify Waxman by 
Hurst to desaturate the fused output image, as claimed, since Waxman teaches that a user to 
allowed to adjust the hue of the output image, and Hurst teaches that, as part of the logistics of 
adjusting the hue of an image signal, it is advantageous to desaturate the image so that changes in 
saturation attributable to the changing of the hue are minimized (see Abstract and column 1, lines 
1 1-28 and column 2, lines 30-38). 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Colin M. LaRose whose telephone number is (571) 272-7423. If 
attempts to reach the examiner by telephone are unsuccessful, the examinees supervisor, Jingge 
Wu, can be reached on (571) 272-7429. The fax phone number for the organization where this 
application or proceeding is assigned is (571) 273-8300. Any inquiry of a general nature or 
relating to the status of this application or proceeding should be directed to the TC 2600 
Customer Service Office whose telephone number is (571) 272-2600. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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